ABSTRACT
IT-Security Tabletop Games for developers have been available in analog format;with the COVID-19 pandemic, interest in collaborative remote security games has increased. In this paper, we propose a methodology to evaluate the impact of a (remote) security game-based intervention on developers. The study design consists of the respective intervention, three questionnaires, and a small open interview guide for a focus group. A validated self-efficacy scale is used as a proxy for measuring effects on participants' ability to develop secure software. We tested this design with 9 participants (expert and novice developers and security experts) as part of a small feasibility study to understand the challenges and limitations of remote tabletop games. We describe how we selected and digitalised three security tabletop games, and report the qualitative findings from our evaluation. Setting up and running the virtual tabletop games turned out to be more challenging and complex for both moderator and participants than we expected. Completing the games required patience and persistence, and social interaction was limited. Our findings can be helpful in building and evaluating a better, more comprehensive, technically sound and issue-specific game-based training measure for developers. The methodology can be used by researchers to evaluate existing and new game designs. © 2022 IEEE.